605074 


(  \ 

ON  SOME  APPLICATIONS  OP  DYNAMIC 

i  PROORAMMINO  TO  MATRIX  THEORY 

! 

I  Richard  Bellman 


V 


^  / 

P-917  ' 


Revised 

November  21,  195^ 


•  p;.'  -'  d  OTS  reJc  i5a 


-lit 


D  D  C 


1700  MAIN  t 


11) 


L  L  1 1  \  /^  [_ 


P-9 17 
Revised 
11-21-56 
-1- 


SUMMARY 


^Tl.e  purpose  of  thi-e  paper  Is  to  dlecuse  some  applications 
tLc  Puiii-tluncii  equation  technique  of  dynamic  programming  to 
some  questions  of  matrix  theory. 


r 

Vfe  shall  first  conelder/the  solution  of  a  system  of  linear 
equa  1 1 ons , 


( 1 )  Ax  -  b  , 

wnere  A  Is  a  Jacobi  matrix.  Then  aimii  jIIscubs  the  same 
problem  lor  the  case  where  A  Is  /almost^  a  block-diagonal  matrix. 
Matrices  of  this  type  arise  In  the  study  of  weak ly— coup  led  mech¬ 
anical  or  electrical  systems.  Finally,  we  dl^ouae  thv« 

calculation  of  the  largest  or  smallest  characteristic  values  of 
matrices  of  this  type. 
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ON  SOME  APPLICATIONS  OP  DYNAMIC  PROORAMMINO  TO  MATRII  THEORY 

fly 

Richard  Ballfflaji 


§1 .  Introduction 

The  purpose  of  this  paper  I0  to  diecuse  eome  applications 
of  the  functional  equation  technique  of  dynamic  programming  to 
eome  quest lone  of  matrix  theory. 

We  shall  first  consider  the  solution  of  a  system  of  linear 
equations , 

(1)  Ax  -  b, 

Mhere  A  Is  a  Jacobi  matrix.  Then  ms  shall  discuss  the  same  problem 
for  the  case  where  A  is  "almost”  a  block-diagonal  matrix.  Matrices 
of  this  type  arise  in  the  study  of  weak i y— coup led  mechanical  or 
electrical  systems.  Finally,  we  shall  discuss  the  calculation  of 
the  largest  or  smallest  characteristic  values  of  matrices  of  this 
type . 


§2.  Jacobi  Matrices 

There  Is  a  large  body  of  literature  connected  with  systems 
of  linear  equations  of  the  form 

•11*1  *12*2  " 

*21*1  *22*2  *25^3  "  ^2* 

*flI*N-l  *N-1  *N'N  *N  "  ^N* 


(1) 
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If  aaaoolatad  matrix  A  la  called  a  Jaoobl 

matrix.  Assuming  that  A  is  positivt  definits,  v«t  Miah  to  obtain 
the  solution  of  this  oystam  in  a  form  quits  dlfferajit  from  any 
of  the  solutions  furnished  by  olaasical  methods. 


Functional  Equations. 


If  A  Is  positive  definite,  the  solution  of  the  system  in 
(1)  is  equivalent  to  that  of  determining  the  mlnlmun  of  the 
Inhomogeneous  form 

N  N 

(1)  Q(3c)  -  I  a.  X  X  -  2  I  b  X  . 

l,J-i  ^  J  1-i  ^  ^ 

Lot  us  define  the  auxiliary  sequence  of  functions 

(2)  r  (z)  -  Uln 

k  -  1,2,  ...,  N,  -  0D<  z  <  GO.  We  wish  to  determine  f^(b^) 
and  the  point  [x^,  x^,  ...,  at  which  the  minimum  is 

attained.  It  is  easy  to  eee  that 


k  k-1 

I  a.  .*4* 4  -  2  I  b.x.  -  2zx 
l,J-i  1  J  1-1  ^  ^ 


(3)  -  Min  [  -  2zx^  ]  -  -z^/a^^. 

*1 

We  now  wish  to  derive  a  recurrence  relation  connecting 
r^(z)  with  f^_^(z).  If  we  fix  x^  and  then  minimize  over  the 
other  x^,  we  obtain 


(^) 


fj^(z)  -  Min 
*k 


r 


k-1 

£ 


_o 


[X^(  l-l,J-l 

^£^  ^1*1  “  “  ®k-l,k*k^*k-l 


k-2 


1 

I 


-  Min 

X. 


r 

L 


a,  1 

kk  k 


2 


-  .  z  X  , 


1  ^k-l,k*k^] 


J 
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Thli  Ijs  an  application  of  tha  "prlnclpi#  of  optimality" 
cf.  [1]. 


§4.  Explicit  Ralatlone . 


W«!)  hava  thu8  reduced  the  determination  of  the  minimizing 

r  ^ 

aequanoe  (x.  ?  to  the  problem  of  computing  the  eaque  nee  'f^(z)f. 

I  ^  ^  j 

With  the  uae  of  digital  computers  and  eystaiaatlc  search  tech¬ 
niques  for  determining  the  location  of  the  minimum,  this  la 
easily  done.  However,  In  this  case,  we  can  determine  the 
sequence  |f^(r)J  in  a  much  more  precise  fashion. 

It  Is  easy  to  see  inductively  that  each  member  of  the 
sequence  a  quadratic  In  z.  Hence  we  set 


(1) 


where  v^^,  w^^  are  Independent  of  z 
Substituting  in  (2.4),  we  have 


(2) 


f.  (z)  -  Min 
*k 


kk''k  - k  '  “k-l  "  'k-l^^k-1  ®k-l,k 


u,,  ,  4-  V 

,2 


'^k-l^^k-1  •k-l,k*k^ 


_  ^  ^  ^k-l*'k-l,k  ■*'  *k-l,k“k-l^k-l 
•kk  ^  •k-l,k  k-1 


Hence 

(5) 
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Using  this  value  of  In  (2),  we  obtain  thus 

recurrence  relatione  connecting  v^,  w^^)  with  l'**k— 1 

(k  )  •  _  - i - ^ - ^ 

®kk  *'k*-l*k-l,k-l 


V 


k 


^k-l*^k-l^k 
*kk  *‘k-l^k-l,k-l 


w 


k 


^ 5 •  Slightly  Intertwined  Systems  » 


2  .2  .2 

-  "k  ^k-1  k-i,  k-1 
- - " —  - 

*kk  '*k-l*  k-1,  k-1 


Let  us  now  consider  the  problem  of  resolving  a  eet  of  linear 
equations  of  the  forme 


(1)  ^  -  0^, 

^21^1  ®22^2  ^2^*3  "  ^2* 

^1^  "  ^2^2  ^ 

bi*^  >  “45*5  *46*6  "  *^4, 

*54*4  *53^5  *56*6  "  ^5' 

*64*4  ^  *65^  ^  *66*6  ^  ^2*7  ’  ^6' 

1>3N,U3N*U5N  *1^>3N,2  +  :5N*2  +  3N 

^  *l-*-3N,3i-5N*:5>3N  -  °li-3N' 


^N-l*3N-3 


^  a 
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1>3J^*1+3N  ■*■  *2-*-3N,2-f3N*2-*-3N  ^  ®2^>3N ,  3-^3W*3^3N  "  °2-« 

^3>3N,l-»-3N^l>3N  ■*■  ^3+3N,2^-3hr*2f:5N  ^  *3>3N,3^3N^3-*-3N  " 

A  (latrix  of  the  type  appearing  above,  we  shall  call  "slightly 
Intertwined."  It  arises  In  a  variety  of  physical,  engineering, 
and  economic  problems  involving  multi— component  systems  with 
weak  coupling. 

In  addition  to  the  question  posed  above,  we  shall  also 
consider  the  eigenvalue  problem.  In  both  cases,  w<%  shall  assume 
that  the  matrix  Is  symmetric,  and.  In  addition,  that  it  is  posi¬ 
tive  definite. 

§6 .  Notation 

Let  us  Introduce  the  matrices 

(0  \ 

for  k  •  0,  1,  2,  ...,  and  the  vectors 

(2)  X  -  *3k-l'  *3k^'  ^  *  ^°3k-2’  °3k-l’  °3k^ 

§7  .  Variational  Fonaulatlon 

Since  the  matrix  of  coefficients  Is,  by  assumption,  positive 
definite,  the  solution  of  the  linear  system  in  (1.1)  la  equivalent 
to  determining  the  minimum  of  the  Inhomogeneous  quadratic  form 

(1)  (x^,  A^x^)  >  (x^,  A^x'-)  ^  ^  (x^,  Aj^x^) 

>  2(c\  x^)  +  2(c^,  x^)  +  ...  4-2(0^,  x^) 

■f  2b^x^x^  4-  ...  4-  2 


2>3N' 
54  3N 
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§8 .  Dynamic  Programming  Fonnulatlon 


For  N  -  1,  2,  and  — ao<z<ao,  lat  ua  Introduc#  the 

B^quence  of  functlono  of  the  variable  z  defined  by 


(1) 


r  N 


N 


f^(z)  -  Min  j  Z  A.x^)  2  Z  (o\  ) 

X,  1  1-1  ^  1-1 

1.  ^ 

H-1  ■] 

We  then  have  the  following  recurrence  relation: 


(2) 


- 


Min  I  (x^,  A.  x^)  ^  2zx 

( X  X  X  )  '  ^ 

v^3N’  3H-1^  3N-2^  L 


3N 


^  2(c  ,  X  )  +  ^K-l  ^^N-l*3N-2  ^ 
Thle  Ifi  an  application  of  the  "principle  of  optimality." 


69-  Computational  Aepecta  —  I 

Since  the  function  f^(z)  la  readily  determined,  we  can 

r 

compute  the  sequence  ^f.  (z)^  by  meana  of  (4.2),  at  the  expense 
of  a  minimization  over  a  >-dlmen8lonai  region.  This  minimization 
may  be  greatly  speeded  up  upon  ualng  the  convexity  properties 
of  the  functions  Involved.  A. though  no  optimal  methods  are  known 
for  mul tl— dlmenalona .  problemo,  the  one— dlmenalonal  method  pre— 
eented  In  [2 j  may  be  employed  In  an  Iterative  manner. 


Writing  (b.2)  In  the  form 


(i) 


fj^(z)  -  Min  Min 

^^H-2  ‘-)^3N"*3N-i 


|^(x^,  ^  ^  2(c^,x^) 


^  ^H-1^^H-i*3N-2^  I' 
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we  Bet  that  It  reduces  to 


(2) 

where 

(3) 


fj^(z)  -  Min 


gj^(*»y)  -  win  (x*^,  Ax^)  +  2zx^j^  2(0^,  x^)  j, 

*5N'*3N-1 

upon  Identifying  x^j^^  as  y.  Thle  new  relation  le  now  well— 
eulted  to  the  technique  described  In  [2] . 

The  computation  of  the  functions  Independent 

of  the  computation  of  the  sequence  |^f^(s)].  Observe  that  this 
computational  approach  Involves  no  divisions. 


§10.  Computational  Aspects  -  II 

Another  approach  to  the  computational  solution  reposts  upon 
the  easily  established  fact  that  le  a  quadratic  In  z  for 

each  N,  1 .e . 

(1)  fy(i)  -  ♦  2V^z  +  W^z^, 

where  U^,  and  are  Independent  of  z.  This  Is  the  same  device 
used  above. 


(2) 


Substituting  In  (8.2),  we  obtain  the  equaclon 

.2 


'*3N-2)  ^ 


N 


Upon  performing  the  minimization  and  determining  the  minimum 
value  of  the  right— nand  side,  we  obtain  recurrence  relations 
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conriectlng  the  triple  (Uj^,  with  the  triple 

V  W  )  . 

H-i'  N-1^ 

Thin  afforde  an  alternative  computational  technique. 

^11.  The  Eigenvalue  Problem 

Coneldor  the  problem  of  determining  the  largest  eigenvalue 
of  the  matrix  appearing  In  (3.I).  Thle  le  equivalent  to  deter¬ 
mining  the  maximum  of 

N  1  N-1 

0)  Q^(x)  -  (X  ,  )  *  2 

^  1  1 

over  the  sphere  S^;  E  (x  ,  x  )  -  1. 

”  J-l 

Def  ne  the  auxiliary  ayetem  of  functlona 
(2)  f^(z)  -  Max  Q^(x)  *•  J  » 

for  — CD  <  z  <  oc^  and  h  -  1,  2,  . 


Then  It  la  readily  aeen  that 


fj^(z)  -  Max  *  (x^,  } 

(x^x^<l 


where 


1 
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